Disparity-Map-Based Rendering for Hole Prevention

A popular way to convert video is given by depth image-based rendering methods, in which a depth that is associated with an image frame is used to generate a virtual view. An adaptive image warping approach as an improvement to the regular approach is proposed. The new algorithm (DMBR) exploits the smoothness of a typical depth map to reduce the complexity of the underlying optimization problem that is necessary to find the deformation, which is required to prevent holes. This is achieved by splitting a depth map into blocks of homogeneous depth using quad trees and running the optimization on the resulting adaptive grid. So finally, we obtain the virtual view of the depth image location on the video image frame. We focus on the problem to generate the virtual videos of different viewpoints from a given video and associated depth maps at a viewpoint so that the viewers can realistically sense the depths. A spatio-temporal global optimization approach is proposed to synthesize the virtual views from a single videoplus-depth video sequence. Because of the lack of knowledge about the 3D structure of a scene and its corresponding texture, the conversion of 2D video, inevitably, however, leads to holes in the resulting 3D image as a result of newlyexposed areas. The new algorithm exploits the smoothness of a typical depth map to reduce the complexity of the underlying optimization problem that is necessary to find the deformation, which is required to prevent holes. Our proposed approach for the pre-processing of depth maps to prevent holes during the view synthesis. We propose to generate the virtual videos of different viewpoints from a given video and associated depth maps at a viewpoint so that the viewers can realistically sense the depths. We also propose a spatio-temporal global optimization approach that formulates an energy function considering depth map, image structure, texture information and patch shift. We also propose a new and efficient adaptation of the regular grid-based warping. Key Terms—2D-to-3D-conversion, depth image-based rendering, hole-filling, image warping, optimization.